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AB3T11ACT 

The present invention provides an OAM tool that enables a network operator to 
trace the path that an Ethernet frame traverses through bridges in a bridge 
Ethernet LAN. Each bridge within the LAN has a control plane which examines a 
query message received from a previous node and determines the next node in the 
route to a destination. The identity of the next node is returned to the soxirce 
together with a time stamp. The process is repeated until the identity of all bridges 
in the pafli has been obtained. 
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ETHERNET ROUTE TRACE 
FIELD OF THE INVENTION 

[0001] This invention relates to data coiwniinications networks and more 
5 particularly to methods and system for tracing the path an Ethernet frame 
traverses in a bridged Ethernet local area network (LAN). 

BACKGROUND 

[0002] The Ethernet system was initially developed to provide communication 
10 between a limited number of stations in a local area network (LAN) environment. 
As transmission medium technology and the related infrastructure have improved 
in recent years the speed at which Ethernet frames can be transported has 
dramatically increased. The distances over which Ethernet frames can be carried 
has also increased with improvements in system architecture. 

15 

[0003] GeneraDy the Ethernet system consists of three basic elements: the physical 
medium to carry Ethernet signals between customer nodes via intermediate 
switches and bridges; a set of medium access control (MAC) rules embedded in 
each Ethernet interface; and an Ethernet frame that consists of a set of bits used to 
20 carry data including control information over the system. 

[0004] Typically, each Ethernet interface, such as a bridge or switch, maintains a 
management information base (MIB) which stores relevant infornrmtion regarding 
each bridge and the identity of other bridges in the system which it can access. 

25 

[0005] There are occasions in which it is desirable, or indeed essential, to be able to 
actually trace or track the route taken by an Ethernet frame as it travels through 
bridges in an Ethernet system. The capability of being able to actually trace the 
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patih that a frame traverses is important in trouble shooting defects in the system 
such as those which might cause excessive packet delays or discrepancies between 
the MIB, control plane and data path (H/W) forwarding tables in Ethernet bridges 
or switches. Furthermore, an Ethernet route trace capability is important for 
5 collecting route statistics necessary for network engineering. 

[0006] There is no known solution to directly trace a data path route through 
Ethernet bridges. 

10 [0007] Some prior art methods find out the path a frame should traverse in a 

bridged LAN by querying the MIB of the bridges. However, the downside to this 
method is that there may be discrepancies between the MIB and the actual path a 
frame traverses (i.e. the data path). These discrepancies could arise because the 
MIB, control plane, and data path tables are out of agreement for some reason, 

15 which could actually be the cause of the problem that is being investigated. 
Therefore, a method that traces the exact path that a frame traverses through a 
bridged LAN is necessary. 

[0008] Another prior art proposal is to perform an Ethemet trace route in a similar 
20 manner to an IP trace route. According to this method frames are repeatedly sent 
along the route, and each successive frame gets one hop closer to the destination 
before a bridge at that current hop responds to the sender of the frame. This 
method is accomplished by sending multicast frames that include a time-to-leave 
(TTL) variable that gets decremented at each hop. When a bridge receives a frame 
25 it decrements the TTL variable, and if the variable is expired the bridge responds to 
the sender. However, the problem with this approach is that the control plane 
(which is software driven) at each hop must process tfie frame, since it is npt 
feasible to upgrade all hardware or Network Processors of bridges in a network to 
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perform this new function. This adds unnecessary delay and therefore any round 
trip measurement would be inaccurate. Furthermore^ any discrepancies between 
the control plane tables and the data path forwarding tables would cause the 
resulting route trace to be different than the actual route that a frame would take 
5 along the data path, i.e. the resulting route trace would be incorrect. 

[0009] Other existing partial proposals require hardware or Network Processor 
dKanges in intermediate bridges to do a trace route. While it may be feasible to 
update edge Network Elements, it may not be feasible to upgrade all core or 
10 intermediate bridges to do this. 

[0010] The challenge is to find a way to do an accurate Ethernet trace route of a 
data frame path without making hardware changes to the bridges in the network. 

15 [00111 Proposals that require hardware or Network Processor changes in 

intermediate bridges to do a trace route are problematic because it may not be 
feasible to upgrade all core or intermediate bridges to support the route trace 
capability. 

20 [0012] Proposals that use the control plane to implement IP-like route traces based 
on TTL would not provide an accurate roimd trip time and could return an 
erroneous route, unless hardware or Network Processors are upgraded to perform 
these functions. 

25 SUMMARY OF THE INVENTION 

[0013] According to the present invention there is provided an OAM (operations, 
administration and management) tool that will enable a network operator to trace 
the path that a frame traverses through bridges in a bridged Ethernet LAN. 
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[0014] According to a first aspect of the present invention there is provided a 
method of tracing a data path route from a source node to a destination node 
through mulHple intermediate nodes in a bridged Ethernet system (a bridge may 
also be connected by non-Ethernet media, e.g. ATM virtual circuits, MPLS Label 
5 Switched Path, IP tunnels or SDH/SONET) comprising: sending a succession of 
Ethernet encapsulated route query messages from the source node, each message 
containing a media access control (MAC) address of the destination node; 
receiving, at route trace enabled bridges in the system, the encapsulated route 
query messages; determining at a control plane of the route trace enabled bridges a 
1 0 MAC address of a next hop bridge on route to the destination node; returning the 
MAC address of the next hop bridge to source node in a response message; 
repeating the sequence through remaining intermediate bridges until a response 
message indicating that the destination node has been identified; and tabulating 
information in the response messages. 

15 

{00151 According to a second aspect of the present invention there is provided a 
system for tracing a data path route from a source node to a destination node 
through multiple intermediate nodes in a bridged Ethernet sjretem comprising: 
means for sending a succession of Ethernet encapsulated route query messages 

20 from the source node, each message containing a media access control (MAC) 
address of the destination node; a control plane at route trace enabled bridges in 
the system to receive the encapsulated route query messages; means at a control 
plane of the route trace enabled bridges for determining a MAC address of a next 
hop bridge on route to the destination node; returning the MAC address of the 

25 next hop bridge to source node in a response message; means for repeating the 
sequence through remaining intermediate bridges until a response message 
indicating that the destination node has been identified; and means for tabulating 
information in the response messages. 



r 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0016] The invention will now be described in greater detail wifli reference to the 
attached drawings wherein: 

5 [0017] Figure 1 illustrates by way of a high level block diagram the architecture of 
the present invention; 

[0018] Figure 2 illustrates an Ethamet frame format; 
10 [0019] Figure 3 illustrates a first trace route query message; 
[0020] Figure 4 illustrates a trace route response message; 
[0021] Figure 5 illustrates a second trace route query message; and 

15 

[0022] Figure 6 is a flow diagram showing the process steps of the present 
invention* 

DETAILED DESCRIPTION OF THE INVENTION 

20 [0023] According to the solution, an operator would initiate an Ethernet traceroute 
from a Provider Edge (PE) device to a destination device. In the present 
description tfie Ethernet traceroute function is known by the abreviation 
Etraceroute. The Etraceroute would return the MAC address (and Bridge 
Identification) of every Bridge on the path to the destination device and the round- 

25 trip delay at every Bridge hop on the way to the destination device. 

[0024] For example an operator would enter the following command: 
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ETraceroute DA <SA> 

[0025] Where SA is the MAC Source Address and DA is the MAC Destination 
Address. If SA is not specified, the source address is set to the MAC Source 
5 Address of the device where the ETraceroute is invoked. 

[00261 Figure lis a high level block diagram of a bridge Ethernet LAN for which 
the Ethernet route trace method is performed. The bridged Ethernet LAN of 
Figure 1 includes two customer equipment nodes (CEl and CE2) communicatively 
10 connected by Ethernet bridges PEl, P2, P3, and PE4 of a provider's network. The 
bridges PEl and PE4 are at the edges of the provider's network, where bridge PEl 
provides network access to CEl and bridge PE4 provides network access to CE2. 
This connectivity may be also shown as follows. 

15 CEl— PEl P2 P3 PE4 CE2 

[00271 In this example, it is assumed that a provider wants to trace the path of a 
data frame from CEl to CE2 of a VLAN with tag value 1000. The operator would 
initiate an Etraceroute from a Provider Edge (PE) device to a MAC address e.g. a 
20 Customer Edge (CE) device as follows: 

ETraceroute CE2_DA CEUSA 

[0028} The Etraceroute should retum the MAC address (and Bridge Identification) 
25 of every Bridge on the path to the destination and the round-trip delay at every 
Bridge hop on the way to the destination. 
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[0029] In general, the Etraceroute message mxist be sent from a PE (PEl) that is the 
Source (PEl) or the next hop of the Source (CEl). At a PE (PEl), an Etraceroute 
Query message for a destination MAC address (e.g. CE2) is created to be sent to the 
next bridge hop (P2) by looking up the MAC forwarding table to the destination. 
5 The Etraceroute message contains the foDowing fields: 

-theDA,i.e, CE2_DA 

-the timestamp when the message is sent out from PEl 

10 [0030] Figure 2 shows an Ethernet frame containing the media access control 
header and the data field. In the following description and in particular with 
reference to Figures 3, 4 and 5 the Ethernet frame format has been modified to 
show only relevant portions as they apply to the present application. Basically the 
traceroute message is a payload and the normal Ethernet header is prepended to 

1 5 the traceroute message. 

[0031] The Etraceroute message is encapsulated in an Ethernet frame (A) with the 
SA set to CEl^A and the DA is set to the MAC Address of P2. The EtherType is 
set to VLAN and the VLAN tag value is set to 1000. The sub EtherType (EtherType 
20 of the frame belonging to VLAN 1000) is set to EtherType^Traceroute. The Ethernet 
frame (A) is then sent to the next hop bridge P2. Figure 3 shows Ethernet frame A. 

[0032] When P2 receives the Etraceroute message, it terminates ttie frame and 
sends the frame to the control plane (or higher layer entity) handling the 
25 EtherType_Traceroute. 



[0033] The control plane Traceroute entity records the time that the query was 
received. It then looks up the next Bridge hop to CE2^DA (this address is in the 
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Etraceroute query message) and creates a Etraceroute response message to PEl 
with the following fields: 

- the timestamp when the message is received 
5 - the next Bridge hop to CE2_DA, i.e. P3. 

[0034] Then P2 encapsulates the Etraceroute response message in an Ethernet 
frame (B) with the SA set to P2, the DA set to the MAC address of PEl, and the 
EtherType set to EtherType.Traceroute, and sends the Ethernet frame (B) to PEl. 
10 Note here it is not necessary for the Etraceroute response message to be encoded 
like the data frame (i.e. the VLAN tag is not required). Ethernet frame B is shown 
in Figxu^e 4. 

[0035] When PEl receives the Etraceroute response message, it terminates the 
15 message (since it is destined to it) and sends the message to the control plane 

handling tiie EtherType_Traceroute. At PEl, anodier Etraceroute Query message is 
created for sending the next bridge hop P3 (obtained in the Etraceroute Response 
message from P2). The Etraceroute Query message contains the following fields: 

20 - theDA,i.e.CE2.DA 

- the timestamp when the message is sent out from PEl 

[00361 The Etraceroute Query message is encapsulated in an Ethernet frame (C) 
with the SA set to CE1_SA and the DA set to the MAC Address of P3, the 
25 EtherType is set to VLAN and the VLAN tag value is set to 1000. The sub 
EtherType (EtherType of the frame belonging to VLAN 1000) is set to 
EtherType_Traceroute. The Ethernet frame (C) is then sent to the next bridge hop 
P3. Ethernet frame C is shown in Figtire 5. 
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[0037] When P3 receives the Etraceroute Query message, it terminates the frame 
and processes the EtherType_Traceroute frame and the whole procedure is 
repeated as shown above for each Bridge hop towards CE2, tmtil a Etraceroute 
Response message is received from PE4. The Etraceroute Response message from 
s PE4 contains the following : 

- the timestamp when the message is received 

- the next Bridge hop to CE2_DA, i-e. NULL. 

10 [0038] When PEl receives a NULL next bridge hop, the EtherType.Traceroute 
entity displays all the collected information as shown below. 

ETraceroute displays all the Bridges as follows: 

15 [0039] ETraceroute from PEl to PE4 for test padcet from CEl to CE2: 

PEltoP2:rtt-10ms 
PEl toP3:rtt-15ms 
PEl toP4:rtt-30ms 
20 PEl to PES: rtt- 40 ms 

[0040] Figure 6 is a flow diagram showing process steps according to the invention. 

[0041] A key aspect of tfie present invention is that the consecutive Etraceroute 
25 query messages are sent to the next hop and the subsequent next hop in the same 
path as a data frame. All bridges ideally should be configured to not discard or 
pimt unknown/ new EtherType such as EtherType^Traceroute to the control plane, 
to prevent intermediate bridges from intercepting EtherType_Traceroute messages. 
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[0042] It should be noted that if the SA is not specified, the Ethernet header SA is 
set to the PFs MAC SA. 

(0043] In this solution, no hardware or Network Processor changes in bridges are 
5 required. Each bridge only need to be loaded with new application software which 
handles the EtherType Traceroute. 

[0044] In a furtiter aspect of the invention, if one of the bridges in the path does not 
have die route trace functionality the following steps are used to skip over that 
10 bridge and continue the trace. The traceroute software at ingress (source node CEl 
or immediate next hop node PEl) would time out when it doesn't receive a 
response from a downstream bridge, and report the trace learned so far (i.e. it can*t 
trace all the way to the destination MAC address). 

1 5 [0045] The ingress bridge may issue another traceroute with the option to multicast 
to downstream bridges. This traceroute multicasts (a multicast address is reserved 
for this purpose) a query message to all downstream bridges (on tiie port towards 
the destination MAC address) and hence shotdd be used sparingly. Etraceroute 
enabled bridges are members of this reserved multicast address. An intermediate 

20 bridge would receive and process the m\dticast query message as well as forward 
the multicast message. If a bridge does not understand the query message it will 
ignore it (but the query message is forwarded to the other dowi\stream bridges of 
the spanning tree). All downstream bridges with forwarding address of the target 
destination MAC address should respond witi\ the next hop bridge MAC address. 

25 

For e.g. 

CEl— PEl F2' — P3 P4 ~P5— PE6~CE2 
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[00461 If P2 and P3 are not Etraceroute enabled, P4, P5 and PE6 respond with the 
appropriate next hops in response to the multicast traceroute query message from 
PEL 

5 [00471 PEl concludes this is the set of consecutive downstream bridges that it can 
trace towards the destination CE2, starting from P4, since each response message 
has a next hop which matches the MAC source address of another response 
message, with the exception of the egress bridge PE6, with a next hop of the 
destination node. PEl then displays the bridges that it can trace, starting from P4 

10 PEl to unknown hop(s) 

PEl to P4 (first Etraceroute aware bridge) : rtt - 30 ms 
PEl to P5:rtt-40ms 
PEl toPE6:rtt^60ms 

15 [00481 If P2 , P3 and P5 are not Etraceroute enabled, P4 and PE6 respond with the 
appropriate next hops in response to the multicast traceroute query message from 
PEl. 

[00491 PEl concludes that there is a number of downstream bridges tiiat it can trace 
20 towards the destination CE2. PEl then displays the bridges that it can trace, 

starting from P4, and any other intermediate bridges that respond to the traceroute 
query message. 

PEl to unknown hop(s) 
25 PEl to P4 (first Etraceroute aware bridge) : rtt - 30 ms 
PEl to unknown hop(s) 
PEl toPE6:rtt-60ms 
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[0050] To improve the accuraq^ of ttie traceroute, the PEl may send (imicast) a 
traceroute query message to all Etraceroute bridges as described before, instead of 
displaying the bridge hops directly after receiving traceroute response message, 
The extra step ensures the traceroute message traverse the paths as a normal data 
5 packet would. The ingress would not send a traceroute query message to 
downstream bridges that have not responded to the multicast query message. 
These bridges would be skipped in the traceroute query and the traceroute 
software at ingress would report no responses from these bridges. 

10 [0051] In the present invention ti\e traceroute message is forwarded like a data 
frame, hence the traceroute correctly and accurately verifies the path and 
functional elements that are forwarding data frames. 

[0052] Further, no hardware or Network Processor changes in bridges are required. 
1 5 Bridges are loaded with new application software that handles the EttierType 
Traceroute. The solution works even if some bridges in the route being traced do 
not have the trace route software installed. 

[0053] In the unlikely event that data path changes occur during the route tracing 
20 procedure, the procedure could be run again, or several more times, in such cases. 
In fact/ multiple tracing for the same route could be a standard option to further 
increase confidence in its results. 

[0054] Although specific embodiments of the invention have been described and 
25 illustrated it will be apparent to one skilled in tiie art ti\at numerous changes can 
be made thereto without departing from the basic concept. It is to be tmderstood, 
however, that such changes will fall within the full scope of the invention as 
defined by the appended claims. 
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I Claim: 

1. A method of tracing a data path route from a source node to a destination 
node through multiple intermediate nodes in a bridged Ethernet system 
comprising: 

sending a succession of Ethernet encapsulated route query messages from 
the source node, each message containing a media access control (MAC) address of 
the destination node; 

receiving, at route trace enabled bridges in the system, the encapsulated 
route query messages; 

determining at a control plane of the route trace enabled bridges a MAC 
address of a next hop bridge on route to the destination node; 

returning the MAC address of the next hop bridge to source node in a 
response message; 

repeating the sequence through remaining intermediate bridges tmtil a 
response message indicating that the destination node has been identified; and 

tabulating information in the response messages. 

2. The method as defined in claim 1 wherein when the encapsulated route 
query messages are received at a non-enabled route trace bridge steps are taken to 
skip to a route trace enabled bridge. 

3. The method as defined in claim 2 wherein the service node sends a mtilti 
cast message to nodes downstream of the non-enabled bridge to locate a route 
trace enable bridge in the route to the destination node. 
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4. Hie method as defined in daim 3 wherein the encapsulated route query 
message is sent to the bridge next to the non-enabled bridge which responds to the 
multi cast message. 

5. The method as defined in daim 1 wherein the query message includes 
address information of the sovirce and destination nodes at connection type. 

6. The metiiod as defined in daim 5 wherein the query message also indudes a 
time stamp value entered by the control plane at respective route trace enabled 
bridges. 

7. The method as defined in daim 1 wherein the response message indudes 
address information of the source nodes and destination node. 

8. The method as defined in daim 1 wherein the step of tabulating information 
generates a report defining bridges traversed by tiie Ethernet frame, 

9. The method as defined in daim 8 wherein time stamp information 
respecting each bridge traversed induded in the report. 

10. A system for tracing a data path route firom a source node to a destination 
node ttirough multiple intermediate nodes in a bridged Ethernet system 
comprising: 

means for sending a succession of Ethernet encapsulated route query 
messages from the source node, each message containing a media access control 
(MAC) address of the destination node; 

a control plane at route trace enabled bridges in the system to receive the 
encapsvdated route query messages; 
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means at a control plane of the route trace enabled bridges for detennining a 
MAC address of a next hop bridge on route to the destination node; 

returning the MAC address of the next hop bridge to source node in a 
response message; 

means for repeating the sequence through remaining intermediate bridges 
until a response message indicating that the destination node has been identified; 
and 

means for tabulating information in the response messages. 
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